(1) Field of the Invention
The present invention relates to a moving picture coding apparatus which performs an orthogonal transform of an input picture by dividing it into two-dimensional blocks and quantizes and codes the data, and more particularly relates to a moving picture coding apparatus which quantizes and codes data of picture by extracting a specific area from the frame of picture in a manner so as to enhance the picture quality of the extracted area.
(2) Description of the Related Art
Recently, visual communications services such as video telephone, video conference etc., show great prospect as an efficient way of using ISDN (Integrated Services Digital Network) and GSTN (General Switched Telephone Network) services. Therefore, investigations into high efficiency coding techniques aiming at efficient transmission of such moving pictures have been intensively carried out. Investigations of these techniques were based on statistic properties of pictures to remove redundancy contained in the pictures to thereby reduce the amount of information. As the coding techniques, the hybrid coding technique is well known which is a combination of motion-compensated prediction and the discrete cosine transform.
However, in the case of transmission at a low bit rate, the quality of the regenerated picture is degraded due to noise generation. Hence, improvement against this has been desired. As a method for improvement of picture quality, a technique has been investigated in which specific areas are extracted from the input picture so that the quantization stepsize is controlled separately for each area. As an example of this, there is an idea for improving the picture quality subjectively, by extracting the face area from the input picture so that areas excepting the face area hereinafter referred to as xe2x80x98backgroundxe2x80x99 will be quantized based on a quantization stepsize greater than that for the face area, to thereby reduce the amount of coding allotted for these areas while a greater amount of coding will be allotted for the face area (for example, R. H. J. M. Plompen, et al.: xe2x80x9cAn image knowledge based video codec for low bitratesxe2x80x9d, SPIE Vol. 804 Advanced in image processing (1987)).
Referring now to FIG. 1, a conventional example using motion compensated prediction and a two-dimensional orthogonal transform will be explained. In FIG. 1, the conventional system includes: a frame memory section 81; a subtracter 82; an orthogonal transformer 83; a quantizer 84; a coding section 85; a buffer memory section 86; an inverse quantizer 87; an inverse orthogonal transformer 88; an adder 89; a frame memory section 90; a motion detecting section 91; a motion compensation predicting section 92; an area extracting section 93; and a coding control section 94.
Now, suppose a picture is input to frame memory section 81. The input picture is a digitized picture obtained from a video camera or the like, and is stored in frame memory section 81 and is divided into blocks of Nxc3x97M pixels (N and M are natural numbers). The picture stored in frame memory section 90 and the picture stored in frame memory section 81 are input to motion detecting section 91 where the motion vectors are detected. In motion compensation predicting section 92, motion compensated prediction values are determined based on the motion vectors and the picture stored in frame memory section 90. The power of the differences between the motion compensated prediction values and the input picture and the power of the input picture are compared to choose interframe prediction or intraframe prediction.
In subtracter 82, the difference between the input picture in frame memory section 81 and motion compensated prediction value from motion compensated predicting section 92 is calculated for each block unit. Orthogonal transformer 83 implements the two-dimensional orthogonal transform of the pixels in each block and outputs the transform coefficients to quantizer 84. Quantizer 84 quantizes the transform coefficients based on the quantization stepsize output from coding control section 94. Coding section 85 implements entropy coding of the quantized output from quantizer 84 to generate coded information. Buffer memory section 86 accumulates the coded information in order to make the system capacity match the transmission rate of the network.
The output from quantizer 84 is also input to inverse quantizer 87, where it is inverse quantized to produce transform coefficients. In inverse orthogonal transformer 88, the transform coefficients are subjected to the two-dimensional inverse orthogonal transform, and the result and the motion compensated prediction value from motion compensation predicting section 92 are added in adder 89. The thus added picture is stored in frames memory section 90 to be used in the next process. Coding controlling section 94 receives valid/invalid information about the face area and the background area as the output from area extracting section 93 and the occupied amount of buffer memory section 86, and determines the quantization stepsize based on these pieces of information. For example, the valid information indicating the face area and the invalid information indicating the background area are output from area extracting section 93 to coding controlling section 94, which chooses a quantization stepsize for the face area smaller than that for the background area, based on the quantization stepsize determined from the occupied amount of buffer memory section 86.
The above technique only sets two levels of quantization stepsize for the face area and the background area, and only specifies the fact that the quantization stepsize for the face area is smaller than that for the background area. Therefore, for actual application of this configuration to a real moving picture coding apparatus, the following operation may and should be done, for instance. Coding controlling section 94 defines dQf, dQb, which are the amounts of displacement for the extracted result from the area extracting section for the quantization stepsize Q which is determined based on the occupied amount of the coded information in buffer memory section 86. Quantizer 84 quantizes the data for the face area using Qxe2x88x92dQf and the data for the background using Q+dQb.
However, in this method, noise etc. existing in the background will also be quantized even though the quantization stepsize is set large, an increased amount of coding arises, which means a reduction in the amount of coding which can be allotted to the face area. Further, since a picture greatly differs from the previous frame when a scene change occurs, if the amount of coding for the background is low, the picture quality of the background area will not become stabilized at an early stage and the reproduced picture becomes rough and ugly, tending to cause block distortion and mosquito noise around the face area due to sharp increase in amount of coding.
Since only the occupied amount of the coded information in buffer memory 86 is referred to when coding controlling section 94 performs control, the quantization stepsize is determined for that buffer""s occupied amount, which makes it difficult to attain the desired, proper, amount of coding. Consequently, variations in picture quality become large between frames, lowering the picture quality.
Moreover, though frames after a scene change, in general, have strong correlation, it is difficult to control quantization so as to attain the target, proper, amount of coding since this system does not use the information from the quantization stepsize of the previous frame or the information as to the amount of coding.
When the amount of displacement of Q (dQf) is kept fixed independent of the size of the face area, the amount of coding per frame may become too high or too low with respect to the target amount of coding depending upon the size of the face area, resultantly the picture quality of the reproduced picture is degraded.
It is therefore an object of the present invention to provide a moving picture coding apparatus which can improve the picture quality by adaptively controlling the quantization and coding in multiple stages after a scene change when a scene change is detected.
In order to achieve the above object, the present invention is configured as follows:
In accordance with the first aspect of the invention, a moving picture coding apparatus includes:
a motion detecting means for detecting motion vectors by dividing the input picture into blocks of Nxc3x97M pixels (N and M are natural numbers) and determining the interframe difference for each block between the current frame and the previous frame;
a motion compensation predicting means for determining motion compensated prediction values from the motion vectors to select interframe prediction or interframe prediction;
an orthogonal transforming means for obtaining transform coefficients by performing an orthogonal transform of the difference between the input picture and the motion compensated prediction values;
a quantizing means for quantizing the transform. coefficients;
a coding means for coding the quantized transform coefficients to generate coded information;
a block skip determining means for determining blocks to skipped without coding, based on the interframe difference, determined by the motion detecting means, between the current frame and the previous frame and based on the determined result of the motion compensated prediction obtained from the motion compensation predicting means;
a specific area extracting means for extracting specific area from the input picture;
a scene change detecting means for detecting a scene change based on the interframe difference;
a coding mode determining means which, when the scene change detecting means has detected a scene change, determines which mode the current state is in among three modes, the scene change frame mode for coding the scene change frame, the post-scene change mode for coding a predetermined number of frames after the occurrence of the scene change, and the specific area mode in which frames downstream are coded to improve the quality of picture of the specific area which has been extracted by the specific area extracting means; and
a coding control means which, based on the mode determined by the coding mode determining means, controls the quantization in a manner such that a scene change frame and a predetermined number of frames downstream are quantized by setting up a quantization stepsize at the quantizing means and by using the quantization stepsize which allows no discrimination between the specific area and the area other than the specific area whilst thinning frames; and frames after the predetermined number of frames are quantized setting up a quantization stepsize at the quantizing means for increasing the coding amount of the specific area whilst restoring the original frame rate, and is characterized in that the coded information is transmitted at a predetermined frame rate by making the system capacity match the transmission rate of the network.
In accordance with the second aspect of the invention, the moving picture coding apparatus having the above first feature is characterized in that the coding control means comprises:
an intraframe coding control means which, in the scene change frame mode, increases the amount of coding per frame by reducing the frame rate and sets up a quantization stepsize based on which the scene change frame is coded by intraframe coding without making any distinction as to the specific area;
a scene change coding control means which, in the post-scene change mode, increases the amount of coding per frame by reducing the frame rate and sets up a quantization stepsize based on which a predetermined number of frames after a scene change are coded using the information of the previous frame without making any distinction as to the specific area;
a specific area enhancement coding control means, which in the specific area mode, sets up a quantization stepsize in accordance with the coding amount information from the coding means, to perform coding based on the information as to skip blocks obtained from the block skip determining means and based on the information as to the specific area extracted by the specific area extracting means; and
a switching means for selecting one of the three control signals based on the mode information from the coding mode determining means.
In accordance with the third aspect of the invention, the moving picture coding apparatus having the above second feature is characterized in that the intraframe coding control means comprises:
a target intraframe-coding amount computing means for calculating the target coding amount of a scene change frame with the frame rate reduced;
a quantization stepsize setting means, which sets up a quantization stepsize based on the target coding amount, using the previously prepared relationship between target coding amounts and quantization stepsizes;
a coding limit monitoring means which, based on the maximum coding amount per frame defined by the standard, monitors whether the amount of coding in the coding means exceeds the coding limit, every predetermined number of blocks, where the coding limit is set up for every predetermined number of blocks;
a quantization stepsize increment means for increasing the quantization stepsize when the amount of coding in the coding means exceeds the coding limit;
a quantization stepsize adding means which adds the quantization stepsize determined by the quantization stepsize setting means and the quantization stepsize which has been increased by the quantization stepsize increment means; and
an average quantization stepsize computing means for calculating the average value of the quantization stepsizes thus added, wherein all the above means are activated when the scene change frame mode information is input from the coding mode determining means.
In accordance with the fourth aspect of the invention, the moving picture coding apparatus having the above second feature is characterized in that the scene change coding control means comprises:
a target scene-change-coding amount computing means for calculating the target amount of coding per frame by reducing the frame rate;
a coding limit monitoring means which, based on the maximum coding amount per frame defined by the standard, monitors whether the amount of coding in the coding means exceeds the coding limit, every predetermined number of blocks, where the coding limit is set up for every predetermined number of blocks;
a coding interrupting means for interrupting the coding of the current frame when the present amount of coding exceeds the coding limit;
a quantization stepsize variational range checking means, which checks whether the input quantization stepsize falls within the specified range, and replaces the quantization stepsize with a closest value thereto falling within the specified range when the quantization stepsize falls out of the specified range, while uses the original quantization stepsize when the quantization stepsize falls within the specified range;
a quantization stepsize adding means which adds the quantization stepsize determined by the quantization stepsize variational range checking means;
an average quantization stepsize computing means for calculating the average value of the quantization stepsizes thus added;
an initial quantization stepsize computing means which computes the initial quantization stepsize based on the average quantization stepsize in the scene change frame and outputs the result to the quantization stepsize variational range checking means; and
a quantization stepsize computing means which computes the quantization stepsize based on the target amount of coding, the information of the amount of coding not exceeding the coding limit and the average quantization stepsize for the previous frame and outputs the result to the quantization stepsize variational range checking means, wherein all the above means are activated when the post scene change mode information is input from the coding mode determining means.
In accordance with the fifth aspect of the invention, the moving picture coding apparatus having the above second feature is characterized in that the specific area enhancement coding control means comprises:
a target specific-area-enhancement coding amount computing means for calculating the target amount of coding per frame;
a target coding amount computing means for specific area allocation which, based on the information as to skip blocks from the block skip determining means, the information as to the specific area extracted by the specific area extracting means and the information as to the target amount of coding from the target specific-area-enhancement coding amount computing means, computes the amount of coding to be allotted for the specific area and that for other than the specific area;
a coding limit monitoring means which, based on the maximum coding amount per frame defined by the standard, monitors whether the amount of coding in the coding means exceeds the coding limit, every predetermined number of blocks, where the coding limit is set up for every predetermined number of blocks;
a coding interrupting means for interrupting the coding of the current frame when the present amount of coding exceeds the coding limit;
a quantization stepsize variational range checking means, which checks whether the input quantization stepsize falls within the specified range, and replaces the quantization stepsize with a closest value thereto falling within the specified range when the quantization stepsize falls out of the specified range, while uses the original quantization stepsize when the quantization stepsize falls within the specified range;
a quantization stepsize adding means which adds the quantization stepsize determined by the quantization stepsize variational range checking means;
an average quantization stepsize computing means for calculating the average value of the quantization stepsizes thus added;
an initial quantization stepsize computing means which computes the initial quantization stepsize based on the average quantization stepsize in the post-scene change mode and outputs the result to the quantization stepsize variational range checking means; and
a quantization stepsize computing means which computes the quantization stepsize based on the target amount of coding, the information of the amount of coding not exceeding the coding limit, the information as to skip blocks from the block skip determining means, the specific area information from the specific area extracting means and the average quantization stepsize for the previous frame and outputs the result to the quantization stepsize variational range checking means,
wherein all the above means are activated when the specific area mode information is input from the coding mode determining means.
In the present invention, the transform coefficients obtained by performing a two-dimensional orthogonal transform of the difference between the input picture and the motion compensated prediction value in each block are quantized and coded to generated coded information, which is output at a transmission rate matching the network. However, when a scene change frame occurs, the picture content of the frame is greatly different from that of the previous frame, so that the information from the previous frame cannot be used, which leads to sharp increase in the amount of coding, resulting in degradation of the picture quality. To deal with such a situation, the coding mode determining means discriminates the state after a scene change into three modes, i.e., the scene change frame mode, the post-scene change mode and the specific area mode. The coding control means controls quantization based on the mode thus determined. Illustratively, in the scene change frame mode and the post-scene change mode, the amount of coding per frame is increased by reducing the frame rate for the scene change frame and a predetermined number of frames downstream and quantized with a quantization stepsize without making any distinction as to the specific area to thereby improve the picture quality. Then, a large amount of coding is allotted to the specific area when the picture quality has become stabilized in the special area mode, to thereby improve the picture quality.